Motivated Learning from Interesting Events: Adaptive, Multitask Learning Agents for Complex Environments

This paper presents a model of motivation in learning agents to achieve adaptive, multi-task learning in complex, dynamic environments. Previously, computational models of motivation have been considered as speed-up or attention focus mechanisms for planning and reinforcement learning systems, however these different models do not provide a unified approach to the development or evaluation of computational models of motivation in different learning settings. This paper models motivation for machine learning as a process that reasons about the states and changes encountered by an agent to produce a learning stimulus that focuses learning and action. Context free grammars and events are introduced as adaptable representations of states and learning tasks. This extends existing learning algorithms to complex, dynamic environments in which tasks cannot be completely predicted prior to learning. Two agent models are presented for motivated reinforcement learning and motivated supervised learning, which incorporate this model of motivation. The formalisms used to define motivated reinforcement learning agents and motivated supervised learning agents further allow the definition of consistent techniques for evaluating motivated learning agent models. Three new metrics are introduced for evaluating learning efficiency, characterizing computational models of motivation and visualizing the emergent behavior of motivated learning agents. The paper concludes with a demonstration of the motivated reinforcement learning agent model that uses novelty and interest as the motivation function. The model is evaluated using the new metrics, showing that motivated reinforcement learning agents using general, task-independent concepts such as novelty and interest can learn multiple task-oriented behaviors.